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The early admonition by Guilford (1952) to avoid the factor analysis of^" 
test scales containing overlapping items has been little heeded by test resear- 
chars. This state of affairs raay be due to the lack of information on the .rf^^ 
distortions or specific influences attributable to item overlap (Share & Rogers j^^.- 
VA,,^ 1965) , and thus such factor studies have continued apace. Significantly/ few ' 
studies directly concerned with the effects on factor structure of item overlap 
have been reported (Rogers & Shure/ 1965) . -"^ 



i^n extensive review of the literature indicates that two main methods 
have been employed in investigating the item overlap issue. The first method 
has been to obtain overlap-free intercorrelations and to compare them, or their 
factor structure ; to original inter cor .relations or their factor structure. The 
underlying assumption in this method is that similarity between the two factor-.-, 
structures indicates that item overlap is not significantly affecting scale 
structure. Four different techniques have been employed in obtaining the 
overlap- Jree intercorrelations: 

1) removal of all overlapping items (Kessebaum £t al_. , 1959); 

2) partial removal of overlapping items (Rogers & Shure, 1965); . 

3) obtaining the square root of the differences between the 
squared common- elements correlations and the squared original 
correlations (Anderson et al , , 1966); and 

4) partialling the item-overlap effect .by' second-order partial 
correlations (Havlicek, 1965) . ,■ 

The studies employing the first three techniques indicated that item ov/jrlap had 
no significant effect on scale structure. The fourth study (Havlicek, .1965) in- 
dicated that item overlap affected the magnitude of the correlations, but unfor- 
tunately no information was available as to the effect on scale structure*. 

The second method has been to obtain the built-in intercorrelations based 
solely on the- scale we%hcs and to compare their factor structure to the factor 
structu:L*e of 'the original intercorrelations. The under lyi.ng assumption in this 
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method is that similarity betv/een the tv;o factor solutions indicates significant 
item ovei-lap effects on scale structure- IX-zo different techniques have been 
employed in obtaining the built in intercorrelation: 

1) computing the common-elements correla.tions between the scales 
(Shure s Rogers, 1965; Silverstein • ■sher, 1968); 

2) assigning random ansv/ers to the scale .\.;:ems and computing the 
intercorrelations from these scores (: .If, 1971). 

The three studies above v;hich have employed this T.ethodhave indicated that factor 
solutions based exclusively on item overlap are highly similar to solutions based 
on original intercorrelations, suggesting that icem overlap is influencing scale 
structure. 

Although factor analysis has boon the most extensively employed technique 
in the analysis of structure among variables, there has recently developed an 
intense amount: of theoretical and applied exploration into more parsimonious and 
nonmetric techniques for the identification of order relations in intercorrelation 
raatrices (Guttman, 19G6; Kruskal, 1964; Shepard, 1962) . These methodologies are 
generally of a multidimensional scaling type that represent variables as points 
in Euclidean space v/ith interpoint distances corresponding to the proximities of 
variables as provided .by their intercorrelations. Thus, geometric representations 
of order relations are provided. A singular feature of these analyses is their 
parsimony where the number of dimensions is concerned. One of the better known 
of these techniques is that of smallest space analysis (SSA) (Guttman, 1966, 1968). 
Kami and Levin (1972) have recently extended the use of the SSA procedure to the 
study of personality scale structure, using intercorrelation matrices of CPI scales* 

Because of the 'parsimony and the geometric representation properties of 
smallest space analyses, and related multidimensional scaling procedures, it may 
be anticipated that their use in the study of personality scale structure will 
increase. However, no consideration of item overlap effects on SSA solutions has 
yet been attempted, though on the basis of the factor analytic work reviewed 
above, it is clear that such research should be undertaken prior to any extended 
application of this tool. It is quite possible that item-overlap increases the 
magnitude of the correlations betv;een scales due to the increase in the range of 
scores, or to the fact that the item has identical scores on both scales. In 
view of this, it might be argued that SSA is to be preferred to factor analysis 
in investigations of item-overlap effects , as SSA uses the order relations of 
intercorrelations and v/ill be unaffected by increases in the magnitude of inter- 
correlations that have no effect on order, while the opposite is true of factor 
analysis. 

The aim of the present study was to compare the original CPI scale struc- 
ture obtained through SSA, v;ith the built-in structure of the CPI on the one hand 
and with the item-overlap-free scale structure on the other hand. 
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Three techniques were employed to obtain the CPI item-overlap-free 
scale structure: (a) intercorrelation matrix oased on reduced scales con- 
taining the non-overlapping items only; (b) intercorrelation matrix based 
on reduced scales containing non-overlapping items and overlapping items 
which were randomly distributed among their scales, so that any overlapping 
item appeared in one scale only; (c) intercorrelation matrix based on 
reduced scales containing non-overlapping items (as in (a) above) with cor- 
rection for reduction in scale length. 

Four matrices were employed for obtaining the CPI built-in structure: 
(a) comraon-elements intercorrelation matrix based solely on the number of 
overlapping items between the scales; (b) intercorrelation matrix based on 
correlations among CPI scales v/hich had been constructed randomly from the 
CPI protocols of real subjects; (c) intercorrelation matrix based on corre- 
lations among CPI scales which had been constructed randomly from CPI protocols 
generated by computer (i.e., "stat rats"). Two such matrices were obtained, 
with each including 153 such "stat subjects". 

The original CPI structure v/as obtained from an intercorrelation matrix 
of the CPI scales which appeared in the CPI manual (Gough, 1957) . A similar 
solution was also obtained from an intercorrelation matrix of original scores 
based on 153 nev; subjects. 

The similarity among the various solutions was assessed both visually 
and through numerical analysis. 

In order to evaluate the similarity among the solutions at the visual 
level, Kami and Levin's (1972) interpretation of the CPI scale structure was 
adopted. Their interpretation fitted the SSA solution based on the original 
scores of the sample used in the present study. Such a result adds further 
support to the reliability of Karni and Levin's solution and indicates as well 
that the sample used in the present report is fairly representative, at least 
where scale structure is concerned. 

The item-overlap-free solutions fitted the same interpretation almost 
completely. The only deviation occurred in the solutions based on reduced 
scores and reduced scores corrected for restricted range, where one scale 
appeared in a different region. Similarly, in the solutions based on the 
number of overlapping items, all solutions closely approximated the original 
solution v;ith only two exceptions, these being obtained in one of the random 
solutions. One numerical analysis of the similarity among the various solu- 
tions was accomplished through Kendall rank correlations (Tau) among the 
derived distances between the scales based on the SSA solutions. These co- 
efficients indicated that the item-overlap-free solutions were closer to the 
original solution than were the built-in solutions. The solution based on 
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partially reduced scales was the closest to the original (Tau = .73) , follov/ed 
by the solution based on reduced scales with correction for length (Tau = .53), 
the solution based on reduced scales without the correction (Tau = ,53), the 
cominon elements solution (Tau = .46), the solution based on scales constructed 
randomly from protocols of real subjects (Tau = .30) and the two solutions 
based on scales constructed randomly from "stat subjects" (Tau « .23 and .20). 
The Kendall Coefficient of Concordance among all solutions was .520, All co-- 
efficients were significant at p < ,001. 

The conf igurational similarity (CS-i) analysis of Lingoes (1967) was 
also employed in comparing the various SSA solutions to the original solution. 
The results here wei?e similar to those above. 

Finally, the inter correlation matrix based on the Kendall rank correla- 
tions among the interpoint distances v/as submitted to SSA. This solution 
indicated the existence of two dimensions among all the solutions. The first 
dimension represented the item-overlap effect, ordering the solutions as 
follows: reduced scales, partially reduced scales, original scales and random 
scales. The second dimension was interpreted as representing a response set 
effect, clustering separately all solutions based on real subjects on one side, 
and random solutions on the other side. 

The high similarity between the item-overlap- free solutions and the 
original solution supports the claim that item-overlap has a negligible effect 
on CPI scale structure- On the other hand, the significant relationship between 
the built-in solutions and the original solution suggests that item-overlap does 
affect CPI scale structure. Such contradictory results are resistant to expli- 
cation. One possible solution might lie in the notion that the number of over- 
lapping items has a special meaning, that is, overlapping of items between scales 
does not just happen, but that there is a reason; if two scales have many items 
in common it is because the two scales are reflecting at least partially similar 
psychological characteristics, and the more items there are in common the greater 
the similarity in the characteristics. Thus the number of common items might be 
an indication of the relationship between scales. 

On the basis of the above argument it might be concluded that overlapping 
items only magnify an intrinsic structure already existing among the scales. 
Such a conclusion, however, should be approached v/ith caution. Overlapping items 
might indicate relationship among scales only in the culture in which the test 
was constructed. Thus, an item overlap effect might distort the scale structure 
when the scale is used in different cultures. In the western culture, for example, 
self -acceptance and d^^minance are characteristics that often go together, therefore 
it is not surprising that the Dominance scale and Self -acceptance scale in the CPI 
share six items in common. Hov;ever, in some eastern cultures sense of self worth 
and self acceptance are independent of one^s interest or ability to dominate 
others. Thus, when used in a culture that is different from the one in which the 
test was constructed, item overlap might impose erroneous relationships among the 
scales. 
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It should be pointed out that the present study was limited to the CPI — 
similar studies employing other tests and inventories^ as v;ell as measures in 
the cognitive domain; should be undertaken. 

Keeping the foregoing limitations in mind/ it might be tentatively con- 
cluded that item overlap and scale interdependency is not as serious a problem 
in personality test construction as some writers have suggested, and that its 
presence or absence may have no significant effect on test structure. 
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Flgy 1. The SSA of the original CPI scales from the CPI manual. 




ERIC 



o 



u 
O 















6 














c: 


a 


a 








o 














o 





?! a 



u 
P4 



o 4: 0 O 
o o o 

O <J o 



rig. Iq. The SSA of the various C?I intercorrelation riatrices, 



